Papers with lightweight classifier
UERLens: Understanding Event Relations in Large Language Models (2026.acl-short)
Copied to clipboard
| Challenge: | Existing studies on event relation extraction (ERE) have focused on improving model performance. |
| Approach: | They propose an interpretability framework for understanding event relations in large language models . they first construct a counterfactual dataset that includes causal, temporal, and sub-event relations . |
| Outcome: | The proposed framework improves event relation extraction by leveraging internal features to train a lightweight classifier. |
Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Low-Confidence Gold (LCG) is a new filtering framework for Large Language Models that curates high-quality subsets while preserving data diversity. |
| Approach: | They propose a new filtering framework that employs centroid-based clustering and confidence-guided selection for identifying valuable instruction pairs. |
| Outcome: | The proposed framework improves performance on a subset of 6K samples while maintaining data diversity. |
Calibrating LLM Confidence by Probing Perturbed Representation Stability (2025.emnlp-main)
Copied to clipboard
Reza Khanmohammadi, Erfan Miahi, Mehrsa Mardikoraem, Simerjot Kaur, Ivan Brugere, Charese Smiley, Kundan S Thind, Mohammad M. Ghassemi
| Challenge: | Despite their impressive performance, large language models (LLMs) consistently struggle with confidence calibration. |
| Approach: | They propose a method to analyze internal representational stability in large language models by applying adversarial perturbations to final hidden states and using a lightweight classifier to predict answer correctness. |
| Outcome: | CCPS significantly outperforms existing methods on LLMs from 8B to 32B parameters in multiple-choice and open-ended formats. |
A Curious Case of Searching for the Correlation between Training Data and Adversarial Robustness of Transformer Textual Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that fine-tuned textual transformer models are vulnerable to adversarial text perturbations. |
| Approach: | They extract 13 different features representing a wide range of input fine-tuning corpora properties and use them to predict adversarial robustness of the fine- tuned models. |
| Outcome: | The proposed framework can be used as an additional tool for robustness evaluation since it saves 30x-193x runtime compared to the traditional technique and can be easily used under adversarial training. |
Safe-FedLLM: Delving into the Safety of Federated Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing work on federated learning for large language models (FL) addresses privacy and data-silo issues in the training of large language model training. |
| Approach: | They propose a probe-based defense framework for FedLLM that constructs defenses across three levels: Step-Level, Client-Level and Shadow-Level. |
| Outcome: | The proposed framework improves FedLLM's robustness against malicious clients while maintaining competitive performance on benign data. |
Trajectory Signatures of Deception in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing interpretability methods exhibit structural limitations in the field of deception detection. |
| Approach: | They propose a method to capture layerwise activations at sparse "decision points" . they capture deception as a dynamic process, a trajectory through the model's hidden-state space . |
| Outcome: | The proposed classifier achieves comparable performance to PCA-reduced probing for binary sycophancy detection and shows preliminary utility for 4-way deception-type classification. |